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Abstract — The recent success of structured solutions for a class 
of information-theoretic network problems, calls for exploring 
their limits. We show that sum-product channels resist a solution 
by structured (as well as random) codes. We conclude that the 
structured approach fails whenever the channel operations do not 
commute (or for general functional channels, when the channel 
function is non decomposable). 
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I. Introduction 

Structured codes can be effective, and significantly better 
than random codes, for various multiuser problems. Promi- 
nent examples include the well known Korner-Marton (KM) 
"modulo-two sum" problem [ 1 ], as well as more recent setups 
such as the "dirty" multiple-access channel (MAC) [2], the 
noisy linear network (along with the compute & forward 
relaying technique) 0, and more 0. The effectiveness of 
structured codes, in particular linear or lattice codes, is due to 
a good match between their linear structure and the additive 
nature of the source or channel involved. 

In the Korner-Marton problem, for example, the two com- 
ponents (X, Y) of a doubly-symmetric binary source are 
encoded separately. The joint decoder is not interested in a 
full reconstruction of X and Y, which are viewed as "helper 
sources", but rather in their modulo-two sum X + Y (or Xor). 
Writing the statistical relation between X and Y as a modulo- 
additive noise channel 

Y = X + Z (1) 

we can recast the problem as that of lossless reconstruction of 
the noise Z from separate encodings of X and Y. 

As shown by Korner and Marton, a linear structured coding 
scheme, which sends the syndromes of X n and Y n with 
respect to a suitable linear binary code, achieves the minimum 
possible rate of H(Z) - the entropy of Z - per each encoder. 
In contrast, a conventional random coding scheme cannot do 
better than to encode at a sum rate equal to the joint entropy 
of X and Y. This corresponds to the Slepian-Wolf solution 
0, i.e., to a lossless reconstruction of both X and Y at the 
decoder. The resulting sum rate can be therefore significantly 
higher than 2H(Z) for highly correlated sources. 

The binary KM problem can be generalized to a g-ary field, 
in which case a linear g-ary code replaces the linear binary 



code in the KM solution (6]. And it also has a quadratic- 
Gaussian version Q. 

A dual example with a similar characteristics is that of the 
"doubly dirty" MAC. This channel extends Costa's "writing 
on a dirty paper" problem |8|, |9| to a MAC; i.e., an additive- 
noise channel with two inputs X\ and X2 and an output Y 
given by 

Y = Xi + X 2 + Si + S 2 + noise (2) 

where Si and S2 are two interferences, each known as "side 
information" to one of the encoders. Addition in (f2]i is over 
some group in the discrete channel case, or the usual addition 
in the continuous case. The problem is made interesting by 
imposing an input constraint upon X\ and X 2 , thus the 
encoders cannot simply subtract the interferences. 

Similarly to the Korner-Marton problem, a linear/lattice pre- 
ceding scheme (which subtracts the interference "modulo the 
code") achieves the capacity region of this channel HUt ElFI 
And in contrast, the rates achieved by a more conventional ran- 
dom binning scheme vanish in the limit of strong interference 
signals. 

This sharp discrepancy is due to the distributive nature of 
the side-information; if the knowledge of Si and S 2 were 
centralized - i.e., they were both known to one encoder or to 
the joint decoder, then random binning could be effective and 
(nearly) achieve capacity; see 0. 

Sometimes structured codes are inferior to random codes. 
This situation occurs when the linear structure of the code 
causes ambiguity at the decoder; for example, the symmetric- 
rates point of the (clean) MAC capacity region, or of the 
Slepian-Wolf rate region. 

In this short note we focus on another, perhaps obvious 
weakness of structured codes: they are sensitive to the structure 
of the channel. Specifically, we show that if the additive 
channel in (HJ or in (f2]) is replaced by a channel involving 
both addition and multiplication, then structured codes - and 
in fact, any other coding scheme - are not effective. 

II. Sum-Product Korner-Marton 

Consider a generalization of the KM problem (1), where the 
statistical relation between the component sources is given by 
the channel 

Y = A + BxC (3) 

1 In the discrete noiseless case the linear coding scheme is exactly optimal, 
while in the continuous case it is asymptotically optimal in the continuous 
high SNR case. 
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Fig. 1. Functional source coding with side information at the decoder. 



where A, B and C are statistically independent, and B ^ 
with probability one. All variables in ([3]) belong to a finite 
field F q of size q, and the + and x are the sum and product 
operations over F q . Here A and Y are viewed as the "helper 
sources", C as the desired source, and B as the "channel state". 
The classical KM problem (03 thus corresponds to the case 
where q = 2, A = X is uniform over {0,1}, B = 1, and 
C = Z. 

In the centralized state case, the channel state B is either 
known to the joint decoder (i.e., one encoder observes A, the 
other encoder observes Y, and the decoder has access to B), 
or to both encoders (i.e., one encoder observes (A,B), and 
the other encoder observes (Y, B)). It is not hard to show that 
in this case the compression rate is H(Z), independent of the 
state distribution, as in the classical KM problem^ 

Our focus is, however, on the de-centralized state case, 
where the channel state B is available only to one encoder. 
The performance in this case can be bounded by the simplified 
setup shown in Fig. 1, where one of the helper sources is 
available (un-coded) as "side information" at the decoder. That 
is, there is only one encoder which observes X = (A,B), 
while Y is available at the decoder, who wishes to reconstruct 
Z = C. 

This latter problem is a special case of functional source 
coding J6), where a function Z = F(X, Y) needs to be 
reconstructed from separate coded versions of X and Y. The 
setup of Fig. Q] corresponds to the case where Y is given un- 
coded as "side information" to the decoder (or it is encoded 
at a rate greater than or equal to H(Y)), and where 



X=(A,B), Y = A + BxC, Z = C 
and F[X,Y) = (Y - A)/B. 



(4) 



A precise definition of functional source coding with side 
information at the decoder is as follows. The encoding function 
is / : F™ — > A4, where the size of the message space A4 is 
2 nR , with n being the code block length and R being the 
coding rate. The decoding function is g : M x F q — > F q . 
The probability of error P e is the probability that g(M, Y n ) 
is not equal to the vector Z n , where Z- L = F(A^,Y), i = 
1 . . . n, and where M — f(X n ) is the encoded message. For 
a given memory less double source (Xi, Yi), [X^, Y2), ••• and 

2 Regarding the former case (B available at the decoder), note that a random 
parity-check matrix H is "good" with high probability for the classical KM 
problem (TJ (i.e., Z n can be reliably decoded from the syndromes HX n and 
HY n ); hence, HB n is "good" with high probability for the generalized KM 
problem {5). In the latter case, the encoders simply divide their observations 
by B, hence get back to the classical KM problem. 



a function F (.,.), a rate R is said to be "achievable" if we can 
make P e as small as desired for some functions / and g, and 
large enough n. Finally, R* denotes the minimum achievable 
rate. 

Clearly, the minimum achievable rate R* satisfies 



H{X\Y) >R*> H{F(X,Y)\Y) 



(5) 



where the LHS corresponds to the case where the decoder fully 
reconstructs X before computing F(X, Y), while the RHS 
corresponds to the case where the encoder also has access to 
Y, so it can first compute F(X, Y) and then compress it. In 
the sum-product case (@), if A is uniform over F q , then the 
bounds © become 



H{B) + H(C) >R*> H(C). 



(6) 



Note that in the classical KM problem B = 1, i.e, H(B) = 0; 
thus the bounds coincide, and the coding rate is merely the 
entropy of the desired variable C. 

Han and Kobayashi [6] give necessary and sufficient con- 
ditions for the LHS of © to be tight0 These conditions are 
satisfied in the sum-product case. 

Lemma 1. In the sum-product (functional source coding) 
problem (@), R* = H(X\Y). Thus, if A is uniform over F q , 
then R* = H(B) + H{C). 

Proof: Follows since two different lines in F q intersect 
in at most one point, implying the condition in [6, lem.l]. ■ 
This result implies that the minimum coding rate R* is in 
general larger than the entropy of the desired variable C, which 
is the rate in the classical KM setting (fl}. In fact, the "extra" 
rate can be as large as log(g — 1), for B which is uniform over 
F q \ 0. 

As a corollary from Lemma [U it follows that the rate of 
the first encoder in the distributed coding setup of d3j is at 
least H(B) + H(C). The interpretation is that the introduction 
of the multiplicative state variable B breaks the symmetry 
of the classical KM problem; B must be fully conveyed to 
the decoder before the linear structure of the channel can be 
utilized (by means of a linear "syndrome" coding) to encode 
the desired source C. 

III. The Sum-Product Dirty MAC 

Consider next a modification of the dirty MAC problem (O, 
in which the channel output is given by 



Y = A + A' + BxC 



(7) 



where as in (f3j) all variables belong to a finite field F q of size q, 
and the + and x are the sum and product operations over F q . 
The inputs of this MAC are A and A' (corresponding to X\ 
and X2 in (f2]i), while B and C are the channel state variables 
(corresponding to Si and S2 in ©). There is no additional 
noise, nor input constraints. 

3 They in fact consider a more general case where both X and Y are 
encoded. 
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Fig. 2. A deterministic channel with two states, one known to the encoder 
and another known to the decoder. 



As in the sum-product KM, the centralized state case is 
easy: if both state variables B and C are known to one encoder, 
or are known to the joint decoder, then the product B xC can 
be simply subtracted; hence the sum capacity is log(q), as if 
the channel was noiseless. 

The interesting setup is, again, the de-centralized state case. 
That is, each encoder has access to only one of the channel 
states, while the decoder is completely ignorant of the states. 
The capacity in this case is bounded from above by that of 
the single-user channel shown in Fig. 2, with 



Y = A + B x C. 



(8) 



Here there is a single encoder that has access to one of the 
states (Si = B), while the decoder has access to the second 
state (S 2 = C), where C is independent of both the input 
X = A and B@ 

A precise definition of encoding and decoding over such 
a channel is as follows. The encoding function is / : M x 
Fq — > F™ and the decoding function is g : F" x F™ — > M, 
where the size of the message space M is 2 ■ , R being the 
coding rate. The error probability P e is the probability that 
g(Y n , S%) ^ M, where Y" depends on X n , S[ l and S%, and 
where X n = f(M, S?) for M e M. A rate R is said to be 
"achievable" if we can make P e as small as desired for some 
functions / and g, and large enough n. Finally, the capacity 
C is the highest achievable rate. 

The sum-product channel ^ is, in fact, a deterministic 
channel, where the output Y is a function of the input X, 
and the two states Si and S 2 : 



Y = F(X, Si,S 2 ). 



(9) 



There is no additional noise in the channel, beyond the 
randomness of the two (known) states Si and S 2 . 

The setup of (0 is an instance of the Gelfand-Pinsker 
problem [9|, i.e., a channel with non-causal side information 
at the encoder. Hence, it has a single letter solution of the 
form 

C = maxI(U;Y,S 2 )-I(U;Si) (10) 

where the maximization is over a suitable set of auxiliary 
random variables U, and functions X = X(U, Si)u 

4 A continuous version of this setup may be thought of as a channel with 
a fading interference II II . 

5 The admissible U's are those for which 52 is independent of (U, X, Si), 
and U <-> (X, Si) f> V form a Mark ov c hain for each value of S2 - Since S2 
is independent of U, the first term in UP! can be written also as I(U; Y\S'2) 



The structure of the function F in © plays a key role 
in determining the capacity C. A favorable case is when F 
has a composite form, where the dependence on the encoder 
variables (X,Si) is separate from the decoder state S%. 

Lemma 2. If the function F can be decomposed into 
F(a,b,c) — F(G(a,b), c), where F is invertible with respect 
to the first argument (i.e., the equation y = F(t,c) has 
a solution t for every y and c), then ( 1701 ) is optimized by 
U = G(X, Si). If also G is invertible with respect to the first 
argument, then the capacity is 

C = log(g) 

and it is achieved by an input p(x\si) that makes G(X,si) 
uniform for all values of s\. 

Proof: The first part follows from lfT2l sec. HLF], and the 
invertibility of F. See also lfl3l . ■ 
The sum-product channel $Q clearly does not satisfy the 
first condition of the lemma, as addition and multiplication do 
not commute. In fact, this channel is much worse. To assess 
its capacity, we shall first establish a relation to a "minimum 
entropy" problem. 

Lemma 3. The capacity of a deterministic two-state channel 
of the form 



Y = X + F(Si,S 2 ) 



(11) 



where Si and S 2 are available at the encoder and the decoder, 
respectively, is given by 



C = \og(q) - inf ^ (fl(ST) + F(S[\ S%)\Sl 



(12) 



where the second term is the (average) conditional entropy 
given S 2 , and where the infimum is over all code block lengths 
n, and functions g : f ™ — > F™. 

Proof: Easy and will be omitted. ■ 
Although ( TT2l is not a single-letter expression, it is some- 
times easier for analysis than the Gelfand-Pinsker solution 
([Tol l. Specifically, for the sum-product channel ^ we have: 

Theorem 1. (Shany-Zamir |14|) For the case where Y = 
X + Si x S 2 , the minimum entropy in ( 1721 ) is bounded by 
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where the upper bound is achieved (for all n) by a quadratic 
per-letter function g(s) — s 2 . 

As a corollary from this theorem, we conclude that the 
capacity of the sum-product channel ([H) is at most one bit, 
for all q. This is quite disappointing when compared to the 
capacity of log(g), which is achievable in the centralized-state 
case. The same statement is true also for the sum-product dirty 
MAC 0. 



IV. Conclusions 

The essence of the examples given in this paper is that the 
order of performing the sum and product operations matters; 
and in fact, they are very "different". One aspect of this 
difference - which is related to the sum-product KM problem 
- is that the expression 

a + b x c (13) 

cannot be decomposed into the form: 
function( function (a, b). c) (not even approximately). A 
second aspect - related to the sum-product dirty MAC - is 
that it is impossible to find a function of a = a(b) such that 
( fT~3~t would be only a function of c (not even approximately). 
In contrast, these two requirements are easily fulfilled if 
the expression in ( TT3l ) is a pure sum a + b + c (by the 
associativity of summation), or a pure product a x b x c (by 
the associativity of multiplication)]^ 

It would be interesting to explore further (and perhaps 
quantify) the information-theoretic aspects of function decom- 
position. Note that this question is almost "distribution free" 
(i.e., nearly independent of the probability distributions of 
sources and channels). A different aspect of "anti structure", 
which is due to a "bad" noise distribution, can be found in 

ma. 
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Note that pure multiplicative versions of the generalized KM and DMAC 
problems can be solved using linear codes over a "logarithmic" domain. 



